Information Theory is a graduate level course (motivated undergraduate students are welcome) designed for students who are interested in the quantitative fundamental limits of information. What is information and how to quantify information? What is the fundamental limits in representing information, delivering information, and learning information? In this course, we try to answer the above high-level questions with mathematical rigor.

We first introduce the fascinating theory originated from Claude E. Shannon, which addresses the above fundamental questions in the context of communication systems. We then provide algorithms that achieve these fundamental limits. Finally, application of information theory to other fields such as statistical inference, secrecy, networks, signal processing, stochastic optimization, etc., will be selectively overviewed in class and explored in final projects.